Towards Automatic Shaping in Robot Navigation

Shaping is a potentially powerful tool in reinforcement learning applications. Shaping often fails to function e ectively because of a lack of understanding about its e ects when applied in reinforcement learning settings and the use of inadequate algorithms in its implementation. Because of these di culties current shaping techniques require some form of manual intervention. We examine some of the principles involved in shaping and present a new algorithm for automatic transferral of knowledge which uses Qvalues established in a previous task to guide exploration in the learning of a new task. This algorithm is applied to two di erent but related robot navigation tasks.